Multiplication Free Holographic Coding

An empirical informational divergence (relative entropy) between two individual sequences has been introduced in [1]. It has been demonstrated that if the two sequences are independent realizations of two finite-order, finite alphabet, stationary Markov processes, the proposed empirical divergence measure (ZMM), converges to the relative entropy almost surely. This leads to a realization of an empirical, linear complexity universal classifier which is asymptotically optimal in the sense that the probability of classification error vanishes as the length of the sequence tends to infinity. if the normalized KL-divergence between the two measures is positive [2]. It is demonstrated that for finite-length sequences that are realizations of finitealphabet, vanishing memory processes with positive transitions, a version of the ZMM [1] is not only asymptotically optimal as the length of the sequences tends to infinity, but is also essentially optimal in the sense that the probability of classification error vanishes if the length of the sequences tends to infinity for a large sub-class of such measures if the length of the sequences is larger than some positive integer N0 . At the same time no universal classifier can yield an efficient discrimination between any two distinct processes in this class, if the length of one of the two sequences N is such that logN < logN0. It is also demonstrated that there are some classification algorithms which are asymptotically optimal as N tends to infinity, but are not essentially optimal. A variable-length divergence measure is defined, that tends to the KL-divergence as N tends to infinity. A new universal classification algorithm is shown to optimal in the sense that the probability of classification error vanishes over the entire class of pairs of finitealphabet, vanishing memory measures with positive transitions, and a positive variablelength divergence, if logN > logN0, while no efficient classification is possible by any universal classifier if logN < logN0.